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ABSTRACT ARTICLE HISTORY 
Despite the prevalence of educational apps for children, there is Received 25 January 2019 
little evidence of their effectiveness for learning. Here, children Revised 19 July 2019 
were asked to learn ten new words in a narrative mobile game Accepted 29 July 2019 
that requires children use knowledge of word meanings to KEYWORDS 

advance the game. Study 1 used a lab-based between-subjects Mobile game; digital game; 
design with middle-SES 4-year-olds and used a receptive vocabu- app; vocabulary; learning; 
lary test to examine whether children learned the game’s words. preschool; classroom; 
Children who played the game answered more questions correctly narrative 

than children who did not play the game. Study 2 used a within- 

subjects design with low-SES preschoolers who played the game 

four times as part of a larger classroom intervention. Children 

showed evidence of learning on both a receptive and an expres- 

sive vocabulary measure. The difference between pre- and post- 

test scores was significantly larger for target words than for five 

non-exposure control words. Results show that both middle-SES 

children in the lab and low-SES children in the classroom learned 

new vocabulary from an interactive mobile game, suggesting that 
developmentally-appropriate mobile games show promise for 

vocabulary learning. 


Technology pervades childhood. A recent survey using a representative sample of more 
than 1,400 U.S. parents found that 98% of children under 8 years old have access to 
a mobile device at home, and that the time children spend on mobile gaming tripled 
between 2013 and 2017 (Common Sense Media, 2017). Apps marketed as “educational” 
are abundant: More than 80% of top-selling apps in iTunes’ Education category target 
children, and more than half of those are in the toddler/preschool category (Shuler, 
Levine, & Ree, 2012). However, very few of these apps were developed with benchmarks 
of educational quality in mind (Hirsh-Pasek et al., 2015; Vaala, Ly, & Levine, 2015) or 
provide developmentally appropriate guidance (Callaghan & Reich, 2018). 
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One domain that could benefit from high quality educational apps is language. Early 
language is an important predictor of later reading ability (NICHD ECCRN, 2005; Storch & 
Whitehurst, 2002). For example, preschool vocabulary skills predict reading comprehension 
in 4th grade (Dickinson & Porche, 2011). Similarly, vocabulary knowledge predicts average 
literacy scores and gains in literacy from 1st to 3rd and from 3rd to 5th grade (Pace, Alper, 
Burchinal, Golinkoff, & Hirsh-Pasek, 2019). Research shows that children from under- 
resourced communities are likely to have slower language development than their middle- 
and upper-SES peers (Durham, Farkas, Hammer, Tomblin, & Catts, 2007; Golinkoff et al., 
2019; Hart & Risley, 1995; Reardon, 2013; von Hippel & Hamrock, 2016). These early delays 
may contribute to disparities in other literacy skills (Beck & McKeown, 2007; Hoff, 2013; 
Muter, Hulme, Snowling, & Stevenson, 2004). Children from disadvantaged backgrounds are 
also likely to spend more time with screen media: Lower-SES children use screen media 
almost 3.5 hours a day compared to less than 2.5 hours for middle-SES children (Common 
Sense Media, 2017). Taken together, these facts highlight a potential role for mobile games 
in supporting early language skills. 

Notably, research suggests that vocabulary instruction occurs infrequently in preschool 
classrooms (Dickinson, Pierre, & Pettengill, 2004). Furthermore, studies suggest that many 
children from low-SES backgrounds do not experience the same rich vocabulary in the 
home environment that is experienced by their middle-SES peers (Huttenlocher, Waterfall, 
Vasilyeva, Vevea, & Hedges, 2010). Thus, mobile games could supplement adult-child 
interactions that support children’s early vocabulary, such as storybook reading. Research- 
based games could become a staple during the over 2 hours a day that children under the 
age of 8 are already spending with screen media (Common Sense Media, 2017). Although 
they lack a strong evidence base, many apps do focus on vocabulary: vocabulary skills are 
the second most common language and literacy skill targeted in apps behind only alphabet 
knowledge (Vaala et al., 2015). 

Research suggests that children can learn new words from e-books read with an adult 
(e.g., Strouse & Ganea, 2017), but less research focuses on independent use of digital 
games. A few studies hint that digital games played on traditional computers may be 
effective for promoting vocabulary learning. In one study, 2"¢ and 3 graders played 
a PBS game called Mission to Planet 429 over a two-week period and showed gains on 
the vocabulary words the game taught compared to a group who played science games 
(Michael Cohen Group, 2010). Similarly, a recent study found that preschoolers and 
kindergartners who played PBS literacy-focused games for 8 weeks at home improved 
on the vocabulary presented in the games relative to children who received puzzle- and 
arts-themed games (Schmitt, Hurwitz, Duel, & Linebarger, 2018). 

These studies were among the earliest in the field and they used computer delivery 
rather than mobile devices. A small amount of preliminary evidence suggests that mobile 
games also support vocabulary learning: Michael Cohen Group (2013) found that Spanish- 
speaking preschoolers who played PBS’s Pocoyo app in their classrooms over a 3-week 
period gained more vocabulary than a control group who played two different games 
designed for bilingual vocabulary learning. In Chiong & Shuler (2010), 3- to 7-year-olds 
played two PBS app-based games at home over a two-week period, including a Martha 
Speaks app that specifically focused on introducing new vocabulary. Children appeared to 
learn new vocabulary from the app, with older children gaining more than 20% on 
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vocabulary words taught in the app from pre- to posttest. However, the study did not 
include any controls against which to compare children’s gains. 

Some lab-based studies may also speak to the potential of mobile games for chil- 
dren’s word learning. For example, Kirkorian, Choi, and Pempek (2016) found that 
toddlers could learn novel object labels from a researcher-designed app in an immediate 
test. Similarly, Russo-Johnson, Troseth, Duncan, and Mesghina (2017) found that 2- to 
4-year-olds learned novel object labels from a simple researcher-designed app in an 
immediate test but that only 4-year-olds transferred their learning to 3D objects. 

Although these findings demonstrate the potential of apps for improving vocabulary, 
there are several important limitations. First, few studies have investigated the use of mobile 
games to support word learning. Although mobile devices are more accessible to children 
because they can be used in many locations, we know little about how their smaller screens 
and touch capabilities may affect learning. Additionally, the two peer-reviewed studies 
assessing word learning from mobile devices only tested immediate learning on object 
recognition measures after a single session of game play. The current studies are the first to 
conduct a rigorous investigation of mobile game use to support preschoolers’ vocabulary 
learning over multiple sessions using multiple stringent measures of learning. 

We created a mobile game to support vocabulary learning based on four principles 
from the science of learning that speak to how to maximize children’s learning from 
apps (Hirsh-Pasek et al., 2015). First, children should be actively involved in a “minds-on” 
way with the material and second, they should be engaged and not distracted. While 
flashy, distracting apps can captivate children they do not necessarily lead to learning 
(Parish-Morris, Mahajan, Hirsh-Pasek, Golinkoff, & Collins, 2013). Third, learning is opti- 
mized when content is embedded in a meaningful context and finally, learning increases 
during social interactions in which a partner responds contingently to the children’s 
actions. Specific examples of how these four principles are incorporated in the game 
designed for this study can be found in the Methods. 

Despite children’s increasing exposure to mobile media and the prevalence of apps 
that claim to be educational, little research has addressed whether preschool children 
can increase their vocabulary through exposure to a mobile game based on principles 
from the science of learning. To address this question, Study 1 used a between-subjects 
design to investigate whether children could learn new vocabulary from the game. The 
goal of Study 1 was to assess proof-of-concept and obtain evidence that would allow for 
causal attributions indicating that children learned the words as a result of playing the 
game. Study 2 used a within-subjects design to extend the research into a more 
naturalistic environment with a specific population of interest, examining learning in 
a classroom setting with children from low-SES backgrounds. 


Study 1 
Method 


Participants 

Participants were recruited by telephone and email from a database of families willing to 
participate in research at a mid-Atlantic university. Participants were predominately white, 
monolingual, and from middle SES backgrounds. Fifty-seven four-year-old children (31 girls) 
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participated (Mage = 53.0 months, SDage = 3.8). Five additional participants were tested but 
excluded due to experimenter error (n = 1), refusal to cooperate (n = 1), scheduling error 
(n = 2), and having less than 70% English spoken at home as indicated by parent 
report (n = 1). 

Both Study 1 and Study 2 were approved by the Institutional Review Board of the 
university and parents provided written informed consent before children entered the 
testing room. All children in Study 1 received a certificate of appreciation and a sticker. 


Procedure 

Children in the game group (n = 34) played the game individually, sitting at a child-sized 
table in a quiet room. The game took 10 to 12 minutes to play through. There was little adult 
guidance or interaction during game play and no training prior to game play; children were 
only told that they would be playing a game. Immediately following the game, children 
completed a receptive vocabulary test. A control group (n = 23) completed the receptive 
vocabulary test after participating in an unrelated study that involved no exposure to the 
words taught in the game. Children were recruited for multiple studies in the lab and their 
group assignment in this study was based on the participant needs across all studies (e.g., 
age, gender, scheduling). Both groups were recruited using the method described above 
and the groups did not differ in gender (p = .88), age (p = .22), mean level of primary 
caregiver's highest level of education (p = .64), or on the percentage of White chil- 
dren (p = .53). 


Target word selection. Words were selected to be difficult and uncommon for this age 
group to ensure that children would be unlikely to know them prior to the study. We 
consulted lists of words categorized based on the grade at which children should know 
or be taught the word (Biemiller, 2010; Chall & Dale, 1995), as well as data from a corpus 
of early childhood naturalistic language on the frequency of word use by children in our 
age range (Baath, 2010). If two of the three sources indicated the word would be too 
easy, it was not included. The ten words in the game included four verbs, four concrete 
nouns, and two abstract nouns. See Table 1. 


Game design. We created the mobile game in collaboration with SmartyPal, a Philadelphia- 
based educational development company; SmartyPal created static cartoon-style images for 


Table 1. Words taught in game with included definitions. 


Form class Word Definition Used in Game 
Concrete noun Spade A type of shovel 

Knoll A small hill 

Hammock* A bed made of cloth or rope that you can hang between two trees 

Scroll* A long, rolled up piece of paper with writing on it 
Abstract noun Misfortune Bad luck 

Nutrition* Food that is healthy for you and helps you grow 
Verb Weep To cry for a while 

Devour To eat food very fast 

Assemble* To put together 

Demolish* To destroy something 


Words marked with an * were included in the analyses for both Study 1 and Study 2; all other words were only used in 
the analyses for Study 1. 
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the game and researchers wrote the game script and logic of gameplay to align with the four 
principles from the science of learning (Hirsh-Pasek et al., 2015). To align with the actively 
involved principle, the game used second-person narration (e.g., “You are going on a space 
adventure”) and a gender-neutral duck as a protagonist to encourage children to be part of the 
action. To align with the engaged principle, our game used attractive, colorful images but did 
not include extraneous hot spots or sound effects unrelated to the vocabulary. These can be 
entertaining but also distracting. 

In the game, the child goes on a space mission to help aliens stuck in quicksand. Aligning 
with the meaningful principle, new vocabulary words were presented as meaningful parts of 
the narrative rather than being presented on a flashcard isolated from any context. For 
example, the protagonist is seen lying in a “hammock” and an alien who broke his leg is 
described as experiencing a “misfortune.” Each word was presented at three separate times 
throughout the game. See Table 2 for examples for the word “hammock.” 

The first time the words were introduced, children heard the definition and were 
asked to repeat the word out loud (a pause in the narration allowed time for the child’s 
response). After all words were introduced, there was a prompt for children to engage 
with the game contingently (i.e., “Tap the rocket ship when you're ready for blast offl”). 
Each word was then presented in a forced choice task, aligning with the active involve- 
ment principle by requiring the child to recall the definition, apply it to a new situation, 
and make a meaningful choice in the story context to answer correctly. For example, 
children had to find the image of a “coop” on a sign that indicated which road to take. 
The game did not advance until children provided a response. For each question, three 
images were presented, and children selected a response by tapping an image. Children 
were given three chances to provide a response and the game provided feedback 
contingent to their response (Kirkorian et al., 2016; Lauricella, Pempek, Barr, & Calvert, 


Table 2. Description of game with examples of images and audio for the target word “hammock”. 


Exposure Description Example audio Example image 
Exposure Word used in context of One afternoon, you were relaxing in ¥ 
1 narrative, definition given, your backyard on a hammock. 
children asked to repeat A hammock is a bed made of cloth 
word out loud or rope that you can hang between 


two trees. Can you say hammock? 


Exposure Question requires children to Find the woman laying on her 


2 use knowledge of word hammock. She'll have a fuel nozzle 
meaning to move game for you! 
forward 


Exposure Word is reviewed in context of First, we found an old lady lying in 
3 reporting back to mission a hammock who gave us a fuel 
control nozzle. 
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2010), aligning with the contingent interactions principle. If children were right on the 
first attempt, they heard an affirmation along with the definition of the word (e.g,., 
“That's right! You found the coop, which is a house where chickens live”). After an initial 
incorrect answer, children heard the definition and were given another chance to 
respond (e.g., “Uh oh, that’s not right. You need to find a coop, which is a house 
where chickens live. Try again!”). This type of scaffolded feedback is rare in commercially- 
available mobile games (Callaghan & Reich, 2018) and may support children’s learning. 
After a second incorrect attempt, children were told to try again, and after a third 
incorrect attempt, children were provided with the correct answer before moving on 
to the next question. Finally, at the end of the game each word was reviewed as part of 
reporting back to mission control. 


Receptive vocabulary measure. A receptive vocabulary measure targeting the 10 
words taught in the game was designed for the study. A receptive measure was chosen 
over an expressive measure because it was quick, easy to administer, and enjoyable for 
children and because we thought that children might be hesitant to verbally define 
words they had only heard in one session of game play. The measure was administered 
on a tablet using an app created for this purpose. Children were first given the 
opportunity to respond to practice items. The words for these items were selected 
based on several data sources indicating that they are quite likely to be known by 
children in this age range (i.e., data from prior studies [Toub et al., 2018], word lists used 
to identify target words [Biemiller, 2010; Chall & Dale, 1995], data on infants’ first words 
[Tardif et al., 2008], and piloting with children in this age range). On both the practice 
items and the test items, children had to select the target from four response images. 
Children heard each word twice - once prior to seeing the response images and then 
again immediately after the response images appeared. For the practice items, children 
were given generic feedback if they responded incorrectly (i.e., “Try again!”). To progress 
onto the test items, children had to respond to two practice items correctly on the first 
attempt. Most children (89%) responded to the first two practice items correctly; three 
children missed one practice item (two in the control condition and one in the game 
condition), two children missed two practice items (both in the game condition), and 
one child (in the control condition) missed three practice items. All children eventually 
answered two items correctly on the first attempt and continued on to the test. In the 
test items, children similarly heard each word once prior to seeing the response images 
and then again immediately after the response images appeared (See Figure 1). The app 
randomized the order of items for each child and randomized the placement of the 
target and foil images on the screen for each item. Two easy filler items were spaced 
evenly throughout the test to help maintain children’s attention and motivation. Most 
children (82%) answered both filler items correctly; eight children missed one filler item 
(four in the control condition and four in the game condition) and one child (in the 
game condition) missed both filler items. No feedback was given for the test or filler 
items. Excluding children who missed more than two practice items or both filler items 
did not change the pattern of results. 

Foil selection was guided by research on lexical development (e.g., Golinkoff et al., 
1995) and included three meaningfully-related foils: thematic (frequently found in same 
event or situation, e.g., a window for the target word awning); conceptual (shares 
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a common category, e.g., a beach umbrella for awning); and phonological (shares 
a similar sound or rhymes, e.g., yawning for awning). This measure was a stringent 
test of word learning, requiring children to 1) generalize beyond the game context as 
the picture used for test was not the same as the picture from the game; and 2) choose 
between meaningfully-related options, such that an incomplete understanding of the 
word’s meaning would be revealed through the use of these purposefully close choices 
(Shirilla, et al., in preparation). 


Results 


Children who played the game answered a significantly higher proportion of test 
questions correctly (MV = .54, SD = .22) than control group children who did not play 
the game (M = .25, SD = .16), t (54) = 6.1, p < .0001, d = 1.6 (see Figure 2). Children who 
played the game also answered a significantly higher proportion of questions correctly 
than would be expected by chance, t (33), p < .0001 d = 1.3, whereas children in the 
control group did not, p = 1. 


Receptive Vocabulary 
/ 


R= 


Score is # of 
words correct 


15 * KX 


—-— RRRK p < .0001 


| +/- 1 standard error 


S 
oo 


S 
a 


S 
ip 


Proportion Correct 


So 
iv 


Control group Game group 


Figure 2. Proportion of words correct on receptive measure for control group compared to game 
group in Study 1. Note: the dashed line represents chance performance. 
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Additional exploratory analyses showed that the number of words for which children 
responded correctly on the first try in the game was correlated at trend level with the 
proportion of receptive questions they answered correctly (r = .47, p = .053), suggesting 
that children who performed better during the game also showed more evidence of 
learning afterwards. 


Discussion 


These results suggest that children can learn new vocabulary from a single bout of playing 
a mobile game. Further, the stringent nature of our learning measure suggests that children 
were able to generalize beyond the game context. This finding is important because 
children are spending increasingly large amounts of time with digital media, and many 
games marketed as educational have little or no research supporting their effectiveness. The 
current findings suggest that mobile games designed based on principles from the science 
of learning could play a role in supporting early vocabulary development. 

However, because the test was immediately following the game, it is not clear 
whether learning would be retained over a longer period of time. Furthermore, children 
were from middle-SES backgrounds and may have more advanced language skills and 
more experience playing educational games and learning new vocabulary than children 
from under-resourced communities. Finally, the game was played in a quiet and mini- 
mally distracting environment, providing optimal learning circumstances, whereas chil- 
dren’s everyday environments include many potential distractions from learning. Study 2 
addresses these limitations by testing the game's effectiveness over a 4-week-long 
intervention with low-SES children in a preschool setting. 


Study 2 


Early childhood education settings have traditionally been slow to incorporate technol- 
ogy (Zevenbergen, 2007). Classrooms present unique challenges for learning from 
mobile games such as increased distraction from peers and having enough devices for 
all children. However, more recent data shows that over half of early childhood teachers 
use tablets in their classrooms at least once a week, but only one third know where to 
find developmentally appropriate content (Center on Media and Human Development, 
2015). Research is needed to examine how high-quality mobile games can become more 
accessible to classroom teachers and how those apps might be incorporated into pre- 
school classrooms in a way that can support teachers’ curricular goals. 


Method 


Participants 

As part of a larger study, participants were recruited from two low-SES preschool programs, 
one in Philadelphia and one in Nashville. Of the ten teachers recruited to participate in the 
larger study, one from each city was randomly assigned to use the mobile game in their 
classrooms. Within those two teachers’ classrooms, 33 three- and four-year-old children 
participated (15 girls; Mage = 52.7 months, SDage = 4.8 months). Based on parent report, the 
majority of the children were African-American (54.5%) and Hispanic (30.3%), with 6.0% 
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White, 3.0% Native American, and 6.0% other. Parents reported highest level of maternal 
education for all but two children: 32.2% some high school, 42.4% high school diploma or 
GED, 18.2% some college, 12.1% trade school or higher. 


Procedure 

Study design. To align with the larger study design (Hopkins et al., 2019), children played 
two versions of the same game: the Quicksand Rescue Mission from Study 1 and the Golden 
Eggs Mission, which had a different storyline about finding a chicken who lays golden eggs. 
Both versions included the five words that we focus on here (two verbs, two concrete nouns, 
and one abstract noun; see Table 1). Each version also taught five additional words that are 
not discussed here because they were also taught in storybook reading activities that were 
part of the larger study design, and thus, we cannot isolate the impact of the game on 
children’s knowledge of those words. Otherwise, the two versions of the game were 
identical in design. Children saw each version of the game twice, so each of the five target 
words was encountered during four gaming sessions. 

This study used a within-subjects design with pre- and post-testing to determine 
whether children could learn the words to which they were exposed. Because we did 
not have a control group of children who did not see the game, we included five no- 
exposure control words in this study. Children were tested on these words at pretest 
and posttest but had no exposure to them in between, so we did not expect children 
to learn them. Control words were drawn from the same pool of difficult words and 
were matched to the game words on difficulty, word type, and frequency in everyday 
speech using several data sources (Baath, 2010, Biemiller, 2010; Chall & Dale, 1995). 
These no-exposure words provide a useful control for any natural learning gains that 
may occur on the target words across this period. 


Classroom procedure. Teachers were asked to pull children from other classroom 
activities (e.g., center time and free play) to play the game individually once a week, 
for a total of four times across the four-week study period. Children played the game in 
a relatively secluded area facing away from the classroom and wore headphones. 
Children played the game independently; teachers were instructed to only help in the 
case of technical difficulties. The app was preloaded with children’s names; teachers 
clicked on a child’s name before game play and the app indicated which children had 
played by placing a star next to their name. 


Measures. Experimenters administered both receptive and expressive vocabulary 
knowledge measures to each child at pre- and posttest. Notably, testing took place 
the week before and the week after the intervention period. Thus, in this study, the 
posttest was not conducted immediately after the final session of game play, represent- 
ing a significant delay between game play and testing. Children were tested on twenty- 
five words in the testing session; we focus here on the five words that were taught in 
both versions of the game and the five no-exposure control words. Order of the 
receptive and expressive measures was randomly assigned at both pretest and posttest. 


Receptive measure. Children were given the same receptive measure as in Study 1. 
Practice item data was missing for one site (20 children) at pre-test. Of the practice item 
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data available, most children responded to the first two practice items correctly (62% at 
pretest, 81% at posttest). At pretest, three children missed one practice item and two 
children missed two practice items. At posttest, five children missed one practice item 
and one child missed seven practice items. Similarly, most children answered both of the 
easy filler items correctly (76% at pretest and 88% at posttest); at pretest, seven children 
missed one filler item and two children missed both, and at posttest, three children 
missed one filler item and one child missed both. Excluding children who missed more 
than two practice items or both filler items did not change the pattern of results. 


Expressive measure. The expressive measure was adapted from Hadley et al.’s (2016) New 
Word Definition Test - Modified (NWDT-M) and was administered by a trained experimen- 
ter. The NWDT-M does not measure the extent to which children give adult-like definitions 
of words, but instead codes for the presence of various elements of semantic and contextual 
information that students provide for each word. The experimenter asked the child to define 
each word (i.e., “What does awning mean?”). Responses were then coded for the following 
information unit categories: perceptual features, superordinate/subordinate, functional infor- 
mation, part/whole, synonyms, antonym, gestures, meaningful context, and basic context. 
See Table 3 for examples and for more information see Hadley et al. (2016). The first four 
categories were used for concrete nouns only. The remaining categories were used for all 
word types. Three coders were trained to distinguish information units and allowed to code 
independently if they reached at least 90% reliability with a gold standard coder on a set of 
five tests. If a coder did not reach 90% reliability, they were given feedback and the process 
was repeated. Throughout coding, the gold standard coder coded every fifth test. If reliability 
was below 90%, all disagreements were discussed until agreement was reached and prior to 
the research assistant continuing independent coding. Analyses were conducted using 
a binary variable where a word was scored as correct if children provided at least one 
information unit. Two pseudo-randomly-created word orders were created; one version was 
used for all children at pretest and the other was used for all children at posttest. 


Table 3. Examples of expressive measure coding. 


Information unit category Description Example 
Perceptual feature How the target looks, smells, tastes, feels, or sounds Knoll: “It’s small” 
(concrete nouns only) 
Superordinate/ Naming a larger category of which the target is Spade: “It's a shovel.” 
subordinate (concrete a member or naming a memory of a category 
nouns only) when the target word is the category 
Functional information Any process, purpose or use of the target Spade: “You can shovel with it.” 
(concrete nouns only) 
Part/whole Describes a distinct part of the target or describes the Spade: “It has a handle.” 
(concrete nouns only) whole that the target is part of 
Synonyms Any word or phrase that is equivalent to the target Weeping: “Crying” 
Antonym A word that is the opposite of the target, plus “not” Devouring: “Not eating slow” 
or other negating word 
Gesture A gesture, action, or facial expressive that shows Devouring: Child mimes quickly 
knowledge of word meaning moving food to their mouth 
and chewing. 
Meaningful context A longer phrase that uses detailed, meaningful Misfortune: “It was a misfortune 
context to explain the target when my toy broke.” 


Basic context (.5 points) | Uses minimal context/typical association but shows Knoll: “There was a grassy knoll.” 
little to no understanding of word meaning 
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Results 


Receptive test 

First, we conducted a mixed effects logistic regression model predicting correct responses 
on the receptive measure at the trial level from session (pre/post) and word type (target/no- 
exposure) controlling for age at pretest and including random intercepts for subjects nested 
within sites (or classrooms given that one classroom at each site used the game). There was 
a significant interaction between session and word type showing the gain from pretest to 
posttest was significantly larger for target words (odd ratio post/pre = 2.56) than for the no- 
exposure control words (odds ratio post/pre = 1.1), B= 0.50, p = .01 (see Figure 3). To further 
examine the change from pre- to posttest on target words, we conducted a similar model 
predicting correct responses on target words only. Significant gains for target words were 
observed on the receptive vocabulary measure from pre-test (M = 0.18, SD = 0.39) to post- 
test (M = 0.38, SD = 0.49), B = 0.53, p < .0001. 


Expressive test 

We could not examine the interaction between session and word type on the 
expressive test because no children answered any control words correctly at pre- 
test. Instead, we examined the simple gains for each word type separately. We 
conducted a mixed effects logistic regression model predicting correct responses 
on target words from session (pre/post) controlling for age at pretest and including 
random intercepts for subjects nested within sites. Significant gains were observed 
from pre-test (V = 0.02, SD = 0.15) to post-test (MV = 0.16, SD = 0.36), B = 1.26, 
p < .001. The parallel analysis for control words showed that children did not show 
substantial improvement for control words from pre-test (M = 0) to post-test 
(M = 0.006, SD = 0.08), (see Figure 4). 


Discussion 


These findings suggest that learning from a mobile game is not limited to exposure 
in a controlled lab environment but can occur in the potentially distracting setting of 
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Figure 3. Proportion of control and target words correct on receptive test at pre-test and post-test in 
Study 2. Note: the dashed line represents chance performance (25%). 
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Figure 4. Proportion of control and play-only words correct on expressive test at pre-test and post- 
test in Study 2. 


a preschool classroom. Strikingly, these differences emerged despite the week-long 
delay between game play and posttests. Furthermore, these findings extended 
beyond receptive knowledge to children’s expressive knowledge of the words’ 
meanings. 


General discussion 


Can preschool-age children learn new vocabulary from mobile games? Despite the 
prevalence of these games in the marketplace and in children’s lives (Common Sense 
Media, 2017), there is remarkably little research to inform our understanding of how 
children may learn new word meanings through game play on mobile devices. The 
current data suggest that mobile games can be used to support preschoolers’ vocabu- 
lary. This finding could have important implications, especially for children from low-SES 
populations who are likely to have slower language development than their middle- and 
upper-SES peers (Hart & Risley, 1995; Golinkoff et al., 2018). Because children from lower- 
SES homes are likely to spend more time with media than children from middle-SES 
households (Common Sense Media, 2017), using mobile games to support vocabulary 
development during their screen time represents an exciting educational opportunity 
and a chance to reduce the gap in early literacy and school readiness. 


Primary findings 


The mobile game created here was designed according to the principles of learning 
science (Hirsh-Pasek et al., 2015). Many mobile games for preschoolers purport to be 
educational (Shuler et al., 2012; Vaala et al., 2015), but research on their effectiveness 
lags behind their rapid adoption. Although this study is an early step in investigating the 
role of mobile games in vocabulary instruction, the current results demonstrate that 
mobile games have the potential to effectively promote vocabulary learning when they 
are designed based on principles from the science of learning. 

These findings provide causal evidence that children can learn vocabulary from a mobile 
game. In Study 1, middle-SES children showed evidence of learning after a single session of 


JOURNAL OF CHILDREN AND MEDIA 13 


game play, compared to children in a control group with no exposure to the game. In both 
studies, preschoolers played the game without adult support and never heard the words 
outside of the game, indicating that word learning occurred in response to the game itself. 
These findings are novel because even the limited previous research in this domain often 
focused on traditional computer games rather than games played on mobile devices 
(Michael Cohen Group, 2010; Schmitt et al., 2018), which are now much more prevalent 
in children’s lives (Common Sense Media, 2017). Furthermore, some previous studies 
demonstrating word learning from mobile devices have used simple apps and novel object 
labels (Kirkorian et al., 2016; Russo-Johnson et al., 2017), whereas the current study shows 
that children can learn a wider variety of words from a narrative-based mobile game that is 
more representative of games likely to be played in children’s everyday lives. 

Study 2 found evidence of learning even in the naturalistic environment of 
a preschool classroom and at a week delay. There are many challenges to incorporating 
educational digital media into school contexts. Teachers may not have access to tech- 
nology and also have many different priorities competing for their attention. Although 
mobile games can be played independently, teachers are responsible for overseeing 
children’s time on the device and ensuring that each child gets an opportunity to play. 
In our study, participation data was captured for one classroom, and all students played 
the game four times as instructed by the intervention protocol, suggesting that teachers 
seemed to effectively incorporate mobile game play into their classroom, at least when 
provided with the technology in the context of a structured intervention. 

Additionally, compared to studies conducted in the lab, schools have the potential for 
more distraction from peers and other activities in the classroom. When children are 
exposed to a mobile game in this environment, there might be reduced attention, 
leading to lower levels of learning. The current results go beyond prior research focused 
on home settings, suggesting that children can learn from mobile games in the class- 
room, despite these challenges. Note that having children wear headphones may be an 
important factor in mitigating distractions. 

Another novel aspect of the current studies is that unlike prior studies using simple 
recognition measures of vocabulary (Kirkorian et al., 2016; Michael Cohen Group, 2013; 
Russo-Johnson et al., 2017), we found gains on a stringent measure of receptive vocabulary 
designed to require children to have deeper understanding of the word meaning (Shirilla, 
et al., in preparation) and on an expressive measure of vocabulary requiring children to 
produce, rather than simply recognize, information about the word. By using these 
rigorous measures, we demonstrate that children’s knowledge gains are more meaningful 
than superficially associating a word with a specific object or a general context, suggesting 
that mobile games can engender deeper knowledge of word meanings. 


Implications 


That children demonstrated vocabulary learning from a mobile game in both studies 
suggests that parents and educators could capitalize on children’s attraction to digital 
media devices for positive effects. In addition to the rapid integration of these devices 
into children’s lives (Common Sense Media, 2017) and recent data suggesting that 
children are more engaged with digital than traditional media (Lauricella, Barr, & 
Calvert, 2014; Richter & Courage, 2017), we saw that, anecdotally, children liked the 
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game used in the current studies, appeared highly engaged during game play, and often 
asked to play it again. Given that engagement and interest are necessary for learning, 
children’s interest in digital devices might be effectively harnessed to support learning. 

Vocabulary, in particular, is an important domain that could benefit from effective 
mobile games, given the disparities between low-SES and middle-SES children’s lan- 
guage skills and home language environments (Durham et al., 2007; Huttenlocher et al., 
2010) and the importance of vocabulary knowledge to children’s later literacy and 
reading skills (Dickinson & Porche, 2011; Golinkoff, Hoff, Rowe, Tamis-LeMonda, & Hirsh- 
Pasek, 2019; Pace et al., 2019). Although screen media should not replace book reading 
with caring adults (Dore, Hassinger-Das, et al., 2018), children can benefit if we capitalize 
on the time they are already spending with these devices to support important skills. 

Mobile games in which children make responses have several advantages over 
passive consumption of media like television. Although the current studies did not 
compare learning from the game to passive exposure, principles from the science of 
learning suggest that children learn best when they are active rather than passive and 
when they have to mentally manipulate ideas (Hirsh-Pasek et al., 2015). Further, because 
mobile games require children to respond to questions or activities, the games also have 
the potential to give adults feedback about children’s progress. For example, teachers 
could receive an email with performance information, indicating which children need 
more support and which words children are struggling to learn. 


Limitations and future directions 


The current studies have several limitations. First, a relatively small number of words 
were taught. Future research should examine the potential of mobile games to support 
learning larger numbers of words. Second, Study 2 used a within-subjects design and 
could not assess how well children would have learned the words from other types of 
instruction. Furthermore, although in Study 2 children retained their knowledge of word 
meanings after a week delay, future research should determine whether gains remain 
over a longer time period. Additionally, because the experimenter or teacher asked 
children to play the game, children may have experienced some extrinisic motivation 
that would not be present in a more naturalistic context. Current research in our lab is 
investigating whether children will also learn from the game in the home environment, 
where it may compete for time and attention with other enticing game options. 

Although we show significant gains on both measures of word learning used in Study 2, 
children did not reach ceiling levels of word knowledge on the receptive test and expressive 
scores were fairly low. However, children played the game only four times across the four- 
week intervention; larger effects may have been observed with a higher dosage. Future 
research should investigate how different levels of dosage, as well as different spacing 
schedules, would influence children’s word learning. Regardless, the effect sizes we 
obtained (posttest target vs. control words: d = 1.37), are consistent with book reading 
interventions (Mol, Bus, & de Jong, 2009). 

Relatedly, although this game was designed based on principles from learning 
science, these principles could translate into mobile games in a variety of ways and 
we did not test different game features individually. For example, we chose to use 
a fantastical context because children find fantasy engaging and motivating (Parker & 
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Lepper, 1992) and some evidence suggests that fantasy contexts may promote vocabu- 
lary learning relative tp realistic contexts (Weisberg et al., 2015). However, other research 
suggests that fantastical contexts might limit children’s transfer of information from 
fiction to reality (Richert & Smith, 2011; Walker, Gopnik, & Ganea, 2015). Similarly, games 
might be more effective if the words being taught were more thematically-related and 
game creators could create a more coherent and meaningful relationship between to-be 
-learned information and the surrounding narrative (Gunter, Kenny, & Vick, 2008). 
Indeed, interventions using thematically-related words have been effective in prior 
research (e.g., Neuman & Dwyer, 2011). Future research might compare children’s 
learning from games with different features such as fantasy content, thematically- 
related words, narrative structure, second-person narration, and anthropomorphized 
animal characters. 

The current studies tested children’s learning from playing a mobile game indepen- 
dently, but children may learn better during joint media engagement with an adult 
(Dore, Hassinger-Das, et al., 2018; Takeuchi & Stevens, 2011). Similarly, children may 
benefit more from the game if they play in dyads, as recent research shows that even 
infants learn language better in the presence of a peer (Lytle, Garcia-Sierra, & Kuhl, 
2018). Studies should investigate whether children would show even stronger learning 
after engaging with a social partner during gameplay. 

Further, these studies investigated children’s learning of vocabulary that they only 
heard in the game. However, mobile games could also be used as a supplement to other 
classroom learning activities (Brittain et al., 2019; Piotrowski, Jennings, & Linebarger, 
2012). For example, a teacher could play a board game supporting vocabulary learning 
with a small group of children while one or more children play the mobile game 
independently to prepare for small group instruction or to review their learning. 
Similarly, mobile games could promote engagement with educational materials outside 
of school. Such a practice could be especially valuable for low-SES families who are less 
likely to have a rich home language environment (Huttenlocher et al., 2010). Future 
research should test the effectiveness of using mobile games in the classroom and at 
home to supplement teacher-led instruction. 


Conclusions 


The current findings suggest that despite concerns about screen time, digital media can be 
used effectively to promote educational aims in young children, especially if media are 
constructed according to science of learning principles (Hirsh-Pasek et al., 2015). Given 
children’s increasing access and exposure to these devices, an indication of their potential 
for learning is promising, especially for children from low-SES families. However, the effec- 
tiveness of one game does not indicate that all educational games are beneficial. The 
current studies demonstrate that apps can be designed to promote learning but, as very 
few apps labeled as “educational” are supported by research (Vaala et al., 2015), we do not 
know how far an app can deviate from learning principles and still be effective. 

It is challenging for researchers to keep up with the seemingly infinite number of 
games available and the rapid pace of development (i.e., both new games and new 
technology such as virtual reality). Continuing to test the effectiveness of generalizable 
game elements, such as the four principles from the science of learning, may help 
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establish guideposts for developers to utilize when designing effective games. It will also 
be important for researchers and media creators to bridge the industry-academia gap to 
ensure that new technologies are developed with these principles in mind (Dore, Shirilla, 
et al., 2018). Furthermore, this research does not address the problem of games being 
deceptively marketed as “educational”. Researchers should consider possible mechan- 
isms for encouraging transparency and rigor in marketing educational apps to children 
and families. For the interested reader, several researchers have discussed how parents 
and educators could use research-based principles to evaluate educational apps (Hirsh- 
Pasek et al., 2015; Papadakis, Kalogiannakis, and Zaranis, 2017). Common Sense Media 
(www.commonsensemedia.org) also includes an evaluation of apps’ educational poten- 
tial in their reviews. 

One thing is certain: children’s use of digital technology is continuing to rise. Children 
from low-SES households may benefit from high-quality educational mobile games, 
given lower levels of home language resources (Huttenlocher et al., 2010) and higher 
levels of media use (Common Sense Media, 2017). Though nothing will replace human 
interactions, researchers may discover that mobile games could support learning across 
a variety of domains. 
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